Image photographing device and control method thereof

ABSTRACT

An image photographing device displays a preview panoramic image and data regarding moving velocity and direction of the image photographing device during panorama photographing in real time. The image photographing device includes a photographing unit to photograph a plurality of images, an image processing unit to convert the images photographed by the photographing unit into preview image data and to generate preview panorama data using the preview image data, and a display unit to simultaneously output a screen to display the preview image data, a screen to display a process of synthesizing the preview panorama data, a moving direction display screen indicating a relative difference between the moving direction of the image photographing device and a predetermined reference direction, and a moving velocity display screen indicating a relative difference between the moving velocity of the image photographing device and a predetermined reference velocity.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Korean PatentApplication No. 2011-0049474, filed on May 25, 2011 in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein by reference.

BACKGROUND

1. Field

Embodiments relate to an image photographing device which provides auser interface to generate a panoramic image, and a control methodthereof.

2. Description of the Related Art

In general, an image photographing device includes a photographing unitto obtain an image. The photographing unit obtains an image formed at afocal length of a lens. The image formed at the focal length of the lenshas a narrower viewing angle range than a viewing angle of a human. Inorder to solve such a problem, a panorama photographing method isproposed. The panorama photographing method is divided into a pluralityof types. For example, there is a panoramic image generation type inwhich a plurality of images is photographed while changing an angle ofthe photographing unit and then the plurality of images are joined inorder to reconstruct one panoramic image.

The image photographing device photographs a plurality of images so asto be joined in the horizontal or vertical direction in a panoramaphotographing mode and then stores the images in a memory. An imageprocessing unit provided in the image photographing device receives theimages stored in the memory. The image processing unit joins theplurality of images into one image. The image processing unit maycorrect chrominance or image deviation generated at an interface betweenthe images. The image processing unit performs image processingoperations, such as aligning, stitching and blending, thus correctingchrominance or image deviation between the plurality of images.

Since the panorama photographing method is complicated in operation andmay require skillfulness compared to a single photographing mode,measures to improve the operation of the panorama photographing methodand to effectively carry out the panorama photographing method may beneeded.

SUMMARY

Embodiments include an image photographing device which outputs apreview panoramic image so as to allow a user to preview a panoramicimage during panorama photographing, and a control method thereof.

Embodiments also include an image photographing device which displaysdata regarding moving velocity and direction of the image photographingdevice during panorama photographing so as to allow a user to easilyoperate the image photographing device, and a control method thereof.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice.

In accordance with one embodiment, an image photographing device, whichperforms panorama photographing by photographing a plurality of imagesand then combining the plurality of images into a single panoramicimage, includes a photographing unit configured to photograph theplurality of images, an image processing unit configured to generatepreview image data from the plurality of images photographed by thephotographing unit and to generate preview panorama data using thepreview image data, and a display unit configured to simultaneouslyoutput a screen to display both the preview image data and a screen todisplay a process of synthesizing the preview panorama data.

The image processing unit may generate the preview panorama data bydownsizing the preview image data and mapping the downsized previewimage data.

The image photographing device may further include a detection unitconfigured to detect a motion vector including data regarding movingdirection and velocity of the image photographing device during panoramaphotographing.

The image processing unit may generate a moving direction display screenindicating a relative difference between the moving direction of theimage photographing device and a predetermined reference direction and amoving velocity display screen indicating a relative difference betweenthe moving velocity of the image photographing device and apredetermined reference velocity, and then output the moving directiondisplay screen and the moving velocity display screen through thedisplay unit.

The image photographing device may further include a control unitconfigured to stop panorama photographing when the moving direction ofthe image photographing device differs from the reference direction byat least a predetermined angle or when the moving velocity of the imagephotographing device differs from the reference velocity by at least apredetermined extent.

The control unit may control the image processing unit so that abnormalperformance of synthesis of the preview panorama data is displayed whenthe moving direction of the image photographing device does not differfrom the reference direction by at least the predetermined angle but theimage photographing device moves in a direction in which the synthesisof the preview panorama data is not normally performed.

The control unit may control the image processing unit so that abnormalperformance of synthesis of the preview panorama data is displayed whenthe moving velocity of the image photographing device does not differfrom the reference velocity by at least the predetermined extent but theimage photographing device moves at a velocity at which the synthesis ofthe preview panorama data is not normally performed.

Photographing modes of the image photographing device may include ageneral photographing mode and a panorama photographing mode, and theimage photographing device may further include an input unit including amode set key configured to select one of the general photographing modeand the panorama photographing mode.

In accordance with another aspect, a control method of an imagephotographing device, which performs panorama photographing byphotographing a plurality of images and then combining the plurality ofimages into a single panoramic image, includes photographing theplurality of images, converting the plurality of images into previewimage data and generating preview panorama data using the preview imagedata, and simultaneously outputting a screen to display both the previewimage data and a screen to display a process of synthesizing the previewpanorama data.

The generation of the preview panorama data using the preview image datamay include generating the preview panorama data by downsizing thepreview image data and mapping the downsized preview image data.

The control method may further include calculating a motion vectorincluding data regarding moving direction and velocity of the imagephotographing device during panorama photographing.

The control method may further include displaying a moving directiondisplay screen indicating a relative difference between the movingdirection of the image photographing device and a predeterminedreference direction.

Panorama photographing may be stopped when the moving direction of theimage photographing device differs from the reference direction by atleast a predetermined angle.

Abnormal performance of synthesis of the preview panorama data may bedisplayed when the moving direction of the image photographing devicedoes not differ from the reference direction by at least thepredetermined angle but the image photographing device moves in adirection in which the synthesis of the preview panorama data is notnormally performed.

The control method may further include displaying a moving velocitydisplay screen indicating a relative difference between the movingvelocity of the image photographing device and a predetermined referencevelocity.

Panorama photographing may be stopped when the moving velocity of theimage photographing device differs from the reference velocity by atleast a predetermined extent.

Abnormal performance of synthesis of the preview panorama data may bedisplayed when the moving velocity of the image photographing devicedoes not differ from the reference velocity by at least thepredetermined extent but the image photographing device moves at avelocity at which the synthesis of the preview panorama data is notnormally performed.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of the embodiments, taken inconjunction with the accompanying drawings of which:

FIG. 1 is a perspective view of an image photographing device inaccordance with one embodiment;

FIG. 2 is a rear view of the image photographing device of FIG. 1;

FIG. 3 is a block diagram illustrating a control system of the imagephotographing device in accordance with the embodiment;

FIGS. 4A, 4B, 4C, 4D, and 4E are views illustrating preview image dataand preview panorama data displayed through a display unit of the imagephotographing device in accordance with the embodiment in order ofphotographing time;

FIG. 5 is a view illustrating screens displaying moving direction andvelocity of the image photographing device in accordance with theembodiment;

FIG. 6 is a view illustrating screens displaying moving direction andvelocity of the image photographing device in a type differing from FIG.5;

FIG. 7A is a view illustrating a moving velocity of the imagephotographing device in accordance with the embodiment;

FIGS. 7B and 7C are views respectively illustrating velocity displayscreens of the image photographing device shown in FIG. 7A according tomovement;

FIG. 8A is a view illustrating a moving direction of the imagephotographing device in accordance with the embodiment;

FIGS. 8B and 8C are views respectively illustrating direction displayscreens of the image photographing device shown in FIG. 8A according tomovement;

FIG. 9 is a flowchart illustrating a preview panorama photographingmethod of the image photographing device in accordance with theembodiment; and

FIG. 10 is a flowchart illustrating a control method of the imagephotographing device in accordance with the embodiment according tomoving direction and velocity, during panorama photographing.

DETAILED DESCRIPTION

Reference will now be made in detail to the embodiments, examples ofwhich are illustrated in the accompanying drawings, wherein likereference numerals refer to like elements throughout.

FIG. 1 is a perspective view of an image photographing device inaccordance with one embodiment, and FIG. 2 is a rear view of the imagephotographing device of FIG. 1.

With reference to FIG. 1, an image photographing device 1 in accordancewith the embodiment includes a shutter button 10 to operate the imagephotographing device, a jog dial 11 to adjust settings of a menu, a modedial 12 to set a photographing mode, a power switch 13 to turn poweron/off, a speaker 14 to output sound, an auto-focus (AF) sub-lightsource 15 to emit light if auto-focus is performed, a remote controllerreceiving unit 17 to receive a signal from a remote controller, amicrophone 16 to which sound is input, a lens 18 to photograph a subjectfor photography, and a viewfinder lens 19 provided to check an imagephotographed by the image photographing device 1 in advance, and a flash20 to emit light.

With reference to FIG. 2, the image photographing device 1 includes aviewfinder 21 to check an image photographed by the image photographingdevice 1 in advance, an auto-focus lamp 22 to represent auto-focus and aflash state lamp 23 to represent a flash state, an LCD button 24 to turnan LCD on and off, a wide angle zoom button 25 to support a wide anglezoom function, a telephoto zoom button 26 to support a telephoto zoomfunction, function buttons 27 to set and cancel various functions, a DCinput terminal 28, an external output terminal 29, a reproduction modebutton 30, an LCD monitor 31, a manual focus button 32, an auto-exposurelock button 33, and a photographed image quality adjust button 34.

The LCD monitor 31 may be an on screen display (OSD) to represent thecurrent photographing mode and state of the image photographing device1, and hereinafter will be referred to as “screen 31”.

FIG. 3 is a block diagram illustrating a control system of the imagephotographing device in accordance with the embodiment.

The image photographing device 1 includes an input unit 100, a lens unit110, a photographing unit 120, an image processing unit 130, a detectionunit 140, a storage unit 150, a control unit 160, and a display unit170.

The input unit 100 includes various function keys shown in FIGS. 1 and2. The input unit 100 may include the mode dial 12 to set thephotographing mode of the image photographing device 1. Here, the modesmay include a general photographing mode and a panorama photographingmode. The input unit 100 outputs a key input signal corresponding to akey, received from a user, to the control unit 160.

The photographing unit 120 includes the lens unit 110 which isprojectable and retractable. The photographing unit 120 may photographimage data through the lent unit 110. The photographing unit 120 mayinclude a camera sensor (not shown) which converts an optical signal,obtained by photographing, into an electrical signal and a signalprocessing unit (not shown) which converts the analog signal, obtainedby the camera sensor (not shown), into a digital signal.

The detection unit 140 includes a motion vector calculation unit 145.The motion vector calculation unit 145 compares a current input image toa previous input image and detects a motion vector of movement of theimage photographing device 1 from a part where chrominance (differencesof RGB values) between the two images occurs. For example, the detectionunit 140 respectively divides the detected two continuous input imagesinto blocks including a plurality of pixels, and detects a motion vectorby comparing RGB values of the pixels in the blocks at the same positionof the two images to each other and calculating differences of colornumbers. However, various methods may be used to calculate the motionvector and this embodiment is not limited to a specific method.

The detection unit 140 may include an illumination sensor to measureillumination and a distance sensor to measure a focal length with asubject for photography.

The image processing unit 130 converts raw image data for each frame,received from the photographing unit 120, into RGB or YUV data, imagesof which may be processed, and performs operations for image processing,such as auto-exposure, white balance, auto-focus, and noise removal. Theimage processing unit 130 compresses the image data output from thephotographing unit 120 using a method set according to characteristicsand size of the display unit 170, or restores the compressed data to theoriginal image data. The image processing unit 130 may have an on screendisplay (OSD) function, and the image processing unit 130 may outputpreview image data according to the size of a screen to be displayed.

The image processing unit 130 generates preview image data and thentemporarily stores the generated preview image data in the storage unit150, in the panorama photographing mode. The image processing unit 130resizes the preview image data temporarily stored in the storage unit150. For example, the image processing unit 130 may downsize the previewimage data from a size of 2040*1280 to a size of 320*240. The imageprocessing unit 130 generates preview panorama data by checking regionsof the resized preview image data overlapped with each other and mappingthe respective preview image data.

The image processing unit 130 transmits a screen on which the previewimage data are displayed and a screen on which the preview panorama dataare displayed so that the two screens are simultaneously output throughthe display unit 170, in the panorama photographing mode. The imageprocessing unit 130 updates the screen on which the preview image dataare displayed and the screen on which the preview panorama data aredisplayed in real time. The image processing unit 130 outputs an imagein which the preview panorama data are synthesized using the previewimage data obtained during movement of the image photographing device 1(for example, an image in which stitching is in progress) in real time.The image in which stitching is in process is an image which is beingcompleted by mapping the plural preview image data.

Here, displaying of the preview image data is displaying of live imagesphotographed in real time on a screen, and displaying of the previewpanorama data is displaying of an image used for stitching duringgeneration of a panoramic image in real time.

The image processing unit 130 outputs screens on which moving directionand velocity of the image photographing device 1 are respectivelydisplayed through the display unit 170 according to the motion vectortransmitted from the detection unit 140, when the photographing unit 120of the image photographing device 1 is moved manually or automaticallyin the panorama photographing mode.

The storage unit 150 includes a program memory and a data memory. Thestorage unit 150 may store various data necessary to control operationof the image photographing device 1 and data selected by a user. Thedata memory stores photographed image data, and the program memorystores a program to control the lens unit 110. The storage unit 150 maystore data corresponding to a zoom stage according to a distance betweenthe subject for photography and the image photographing device 1. Thestorage unit 150 may store the preview image data output from the imageprocessing unit 130 and the preview panorama data generated throughcombination of the preview image data.

The control unit 160 generally controls operation of the respectiveunits. The control unit 160 receives an external signal through thephotographing unit 120, processes the external signal, and outputs imageoutput signals necessary to perform various operations, including thephotographed image, through the display unit 170.

The control unit 160 switches from the photographing mode of the imagephotographing device 1 to the panorama photographing mode, when a userselects the panorama photographing mode through the input unit 100. Inthe panorama photographing mode, a plurality of images are photographedwhile changing the angle of the photographing unit 120 and the pluralityof images are joined to reconstruct one panoramic image. Here, theplurality of images may be photographed in continuous moving imagetypes.

The control unit 160 controls the image processing unit 130 according todata regarding the motion vector transmitted from the detection unit140, thus adjusting the velocity display screen and the directiondisplay screen output from the display unit 170.

The control unit 160 extracts velocity data from the data regarding themotion vector transmitted from the detection unit 140, completesoperation of the image photographing device 1 in the panoramaphotographing mode, when the moving velocity of the image photographingdevice 1 is greater than a reference velocity by a predetermined extentor more, and then stops output of the preview panoramic image. Thereference velocity is the optimum velocity for normal photography duringphotographing in the panorama photographing mode, and is set by adesigner.

On the other hand, when the moving velocity of the photographing unit120 does not allow normal performance of the panoramic image synthesisbut is not greater than the reference velocity by the predeterminedextent or more during photographing in the panorama photographing mode,a preview panoramic image in which stitching is not normally performedis output through the display unit 170. The user confirms that thepreview panoramic image is not normal, and then lowers the movingvelocity.

The control unit 160 extracts moving direction data from the dataregarding the motion vector output from the detection unit 140, andcompletes operation of the image photographing device 1 in the panoramaphotographing mode, when the moving direction deviates from a referencedirection by a predetermined angle or more, and then stops output of thepreview panoramic image through the display unit 170. Here, thereference direction is the optimum direction to be moved for panoramaphotographing. The reference direction is set by a designer, and isdesigned so as to be varied according to panoramic image generationtypes. For example, there are a planar panoramic image generation typeand a cylindrical panoramic image generation type. The optimum movingdirections of the image photographing device 1 corresponding to variouspanorama generation types are considered during design, and the optimummoving direction may be set to the reference direction. Here, setting ofthe reference direction does not mean setting of the absolute direction,but means setting of the reference direction corresponding to thepanoramic image generation type. For example, in the case of the planarpanorama generation type, the initial moving direction may be set to thereference direction. Hereinafter, the planar panoramic image generationtype will be exemplarily described for clear description of theembodiment. However, the embodiment is not limited to the planarpanoramic image generation type.

The display unit 170 may simultaneously display both the screen on whichthe preview image data are displayed and the screen on which the previewpanorama data are displayed during panorama photographing.

FIGS. 4A, 4B, 4C, 4D, and 4E are views illustrating preview image dataand preview panorama data displayed through the display unit of theimage photographing device 1 in accordance with the embodiment in orderof photographing time.

Hereinafter, from among the planar panoramic image generation type, ascheme in which multiple scenes are arranged in a line in the horizontaldirection will be exemplarily described.

When a user selects the panorama photographing mode through the inputunit 100, the control unit 160 switches from the current photographingmode of the image photographing device 1 to the panorama photographingmode. When the current photographing mode of the image photographingdevice 1 is switched to the panorama photographing mode, the controlunit 160 causes a moving picture input through the photographing unit120 to be converted into a panoramic image by the image processing unit130 and then outputs the panoramic image through the display unit 170.

The image photographing device 1 photographs the moving picture togenerate panorama data according to manual movement by the user in thepanorama photographing mode. The photographing unit 120 transmits thephotographed moving picture for each frame to the image processing unit130, and the image processing unit 130 converts the received image datafor each frame into data, images of which may be processed, and outputspreview image data according to the size of a displayed screen.

The preview image data output from the image processing unit 130 areoutput as a live view through the display unit 170, and are transmittedto and stored in the storage unit 150. The image processing unit 130resizes (precisely, downsizes) the preview image data stored in thestorage unit 150. The image processing unit 130 generates previewpanorama data by checking regions of the resized preview image dataoverlapped with each other and mapping the respective preview imagedata. The image processing unit 130 outputs partial images, which arestitched, during generation of the preview panorama data in real timethrough the display unit 170.

FIG. 4A is a view illustrating the screen which starts to output thepreview image data and the preview panorama data, generated by the imageprocessing unit 130, through the display unit 170, and FIGS. 4B to 4Eare views illustrating a process of stitching the preview panorama datawhen the image photographing device 1 moves in the horizontal directionafter the state of FIG. 4A. With reference to FIG. 4A, a region ‘a’ isphotographed, obtained preview image data are displayed as a live view,and preview panorama data of the region ‘a’ are displayed on a screen 31b. With reference to FIG. 4B, the image photographing device 1 movesfrom the position shown in FIG. 4A in the horizontal direction and thusa region ‘b’ is displayed as a live view on a screen 31 a, and a scenein which newly obtained preview panorama data of the region ‘b’ arestitched to the previously obtained preview panorama data of the region‘a’ is displayed on the screen 31 b. FIGS. 4C to 4E illustrate displayof preview image data of regions ‘c’ to ‘e’ on the screen 31 a andprocesses of stitching preview panorama data of the regions ‘c’ to ‘e’on the screen 31 b, simultaneously.

As shown in FIGS. 4A to 4E, the screen output through the display unit170 is divided into the screen 31 a displaying the preview image dataand the screen 31 b displaying progress of stitching of the previewpanorama data, thus simultaneously displaying the screen 31 a and thescreen 31 b.

FIG. 5 is a view illustrating screens displaying moving direction andvelocity of the image photographing device in accordance with theembodiment.

The motion vector calculation unit 145 compares an image currently inputto the image photographing device 1 to a previously input image, anddetects a motion vector of movement of the image photographing device 1from a part where chrominance between the two images occurs. The motionvector includes data regarding the moving direction and the movingvelocity of the image photographing device 1.

The image photographing device 1 may display a predetermined referencevelocity and a current moving velocity relative to the referencevelocity through a velocity display screen 200, during panoramaphotographing. With reference to FIG. 5, reference numeral 210 indicatesa curve of the reference velocity and reference numeral 220 indicates acurve of the current moving velocity relative to the reference velocity.When the current moving velocity curve 220 is above the referencevelocity curve 210 in the velocity display screen 200 of FIG. 5, thecurrent moving velocity is higher than the reference velocity, and whenthe current moving velocity curve 220 is below the reference velocitycurve 210, the current moving velocity is lower than the referencevelocity. A user may comparatively easily adjust the moving velocity ofthe image photographing device 1 with reference to the velocity displayscreen 200 output through the display unit 170.

The image photographing device 1 may display a predetermined referencedirection and a current moving direction relative to the referencedirection through a direction display screen 300, during panoramaphotographing. With reference to FIG. 5, reference numeral 310 indicatesa curve of the reference direction and reference numeral 320 indicates acurve of the current moving direction relative to the referencedirection. When the current moving direction curve 320 is above thereference direction curve 310 in the direction display screen 300 ofFIG. 5, the image photographing device 1 moves upwards from thereference direction, and when the current moving direction curve 320 isbelow the reference direction curve 310, the image photographing device1 moves downwards from the reference direction.

FIG. 6 is a view illustrating screens displaying moving direction andvelocity of the image photographing device in a type differing from FIG.5.

In FIG. 6, the velocity display screen 200 and the direction displayscreen 300 are displayed in a gauge type.

In the gauge type velocity display screen 200, a velocity indicatinggauge may be varied according to design. Here, the velocity indicatinggauge may be designed such that the 12 o'clock direction represents areference velocity, when an indicator 230 moves to the right from the 12o'clock direction, the current moving velocity is higher than thereference velocity, and when the indicator 230 moves to the left fromthe 12 o'clock direction, the current moving velocity is lower than thereference velocity. (Here, the velocity indicating gauge may be designedsuch that, when the indicator turns in the clockwise direction andpasses by the 6 o'clock direction or when the indicator turns in thecounterclockwise direction and passes by the 6 o'clock direction,panorama photographing is stopped).

In the gauge type direction display screen 300, a direction indicatinggauge may be varied according to design. Here, the direction indicatinggauge may be designed such that the 12 o'clock direction represents areference direction, when an indicator 330 moves to the right from the12 o'clock direction, the image photographing device 1 moves upwardsfrom the reference direction, and when the indicator 230 moves to theleft from the 12 o'clock direction, the image photographing device 1moves downwards from the reference direction.

Although FIGS. 5 and 6 exemplarily illustrate the velocity displayscreens and the direction display screens, the embodiment is not limitedthereto. That is, any screens which display moving velocity anddirection of the image photographing device 1 during panoramaphotographing may be applied to the embodiment.

The image photographing device 1 completes panorama photographing, whenthe moving direction of the image photographing device 1 deviates fromthe reference direction by a predetermined angle or more during panoramaphotographing, and completes panorama photographing, when the movingvelocity of the image photographing device 1 differs from the referencevelocity by a predetermined extent or more.

FIG. 7A is a view illustrating a moving velocity of the imagephotographing device 1 in accordance with the embodiment, and FIGS. 7Band 7C are views respectively illustrating velocity display screens 200of the image photographing device 1 shown in FIG. 7A according tomovement.

With reference to FIG. 7A, the image photographing device 1 moves at avelocity of 7 cm/sec by a user during panorama photographing. Here, ifit is assumed that the reference velocity is 5 cm/sec, the currentmoving velocity of the image photographing device 1 exceeds thereference velocity by +2 cm/sec, and thus FIGS. 7B and 7C illustrate astate in which moving velocity of the image photographing device 1exceeds the reference velocity by +2 cm/sec.

The image photographing device 1 may be designed such that panoramaphotographing is stopped when the moving velocity of the imagephotographing device is more than 100 cm/sec.

FIG. 8A is a view illustrating a moving direction of the imagephotographing device 1 in accordance with the embodiment, and FIGS. 8Band 8C are views respectively illustrating direction display screens 300of the image photographing device 1 shown in FIG. 8A according tomovement.

With reference to FIG. 8A, the image photographing device 1 moves in adirection upwards from the reference direction by an angle of +30°during panorama photographing. With reference to FIGS. 8B and 8C, therespective direction display screens 300 illustrate that the movingdirection of the image photographing device 1 deviates from thereference direction by an angle of +30°. A user may easily adjust themoving direction of the image photographing device 1 with reference tothe direction display screen 300.

With reference to FIG. 8A, the image photographing device 1 may bedesigned such that panorama photographing is stopped when the movingdirection of the image photographing device deviates from the referencedirection by an angle of ±90° or more. However, such an angle at whichpanorama photographing of the image photographing device 1 is stopped(i.e., an angle by which the moving direction of the image photographingdevice 1 deviates from the reference direction) may be set by a designeror a user.

FIG. 9 is a flowchart illustrating a preview panorama photographingmethod of the image photographing device 1 in accordance with theembodiment.

When a user selects the panorama photographing mode through the inputunit 100, the control unit 160 of the image photographing device 1switches from the current photographing mode of the image photographingdevice 1 to the panorama photographing mode. When the currentphotographing mode of the image photographing device 1 is switched tothe panorama photographing mode, the control unit 160 causes a movingpicture input through the photographing unit 120 to be converted into apanoramic image by the image processing unit 130 (Operations 400 and410).

The image processing unit 130 converts raw image data for each frame,received from the photographing unit 120, into RGB or YUV data, imagesof which may be processed, and performs operations for image processing,such as auto-exposure, white balance, auto-focus, and noise removal. Theimage processing unit 130 sets the data output from the photographingunit 120 according to characteristics and size of the display unit 170,and outputs preview image data (Operation 420).

The image processing unit 130 generates the preview image data and thentemporarily stores the generated preview image data in the storage unit150. The image processing unit 130 resizes the preview image datatemporarily stored in the storage unit 150. The image processing unit130 generates preview panorama data by checking regions of the resizedpreview image data overlapped with each other and mapping the respectivepreview image data (Operation 430).

The image processing unit 130 simultaneously outputs a screen on whichthe preview image data are displayed and a screen on which the previewpanorama data are displayed. That is, the image processing unit 130outputs a process of synthesizing the preview panorama data using thepreview image data obtained during movement of the image photographingdevice 1 in real time (Operation 440).

FIG. 10 is a flowchart illustrating a control method of the imagephotographing device 1 in accordance with the embodiment according tomoving direction and velocity, during panorama photographing.

When a user selects the panorama photographing mode through the inputunit 100, the control unit 160 of the image photographing device 1switches from the current photographing mode of the image photographingdevice 1 to the panorama photographing mode. When the currentphotographing mode of the image photographing device 1 is switched tothe panorama photographing mode, the control unit 160 causes a movingpicture input through the photographing unit 120 to be converted into apanoramic image by the image processing unit 130 (Operations 500 and510).

The motion vector calculation unit 145 of the detection unit 140compares a current input image to a previous input image and detects amotion vector of movement of the image photographing device 1 from apart where chrominance (differences of RGB values) between the twoimages occurs (operation 520).

The image processing unit 130 outputs the velocity display screen andthe direction display screen through the display unit 170 by combinationbetween data regarding the reference velocity and direction inputthrough the control unit 160 and data regarding the current movingvelocity and direction of the image photographing device 1, duringpanorama photographing (Operation 530).

The control unit 160 confirms the moving velocity of the imagephotographing device 1, and completes panorama photographing when themoving velocity of the image photographing device 1 differs from thereference velocity by a predetermined extent or more (Operations 540 and560).

The control unit 160 confirms the moving direction of the imagephotographing device 1, and completes panorama photographing when themoving direction of the image photographing device 1 differs from thereference direction by a predetermined angle or more (Operations 550 and560).

The above-describe reference velocity and direction may be values variedaccording to detailed factors of the panorama photographing mode and beset by a designer or a user. Further, the reference velocity anddirection are not limited to designated velocity and direction but maybe varied in real time according to the detailed factors of the panoramaphotographing mode. Here, values displayed on the velocity displayscreen and the direction display screen indicate relative differenceswith the reference velocity and direction, and thus the referencevelocity and direction may be varied.

As is apparent from the above description, an image photographing deviceand a control method thereof in accordance with one embodiment may checka preview panoramic image in real time during panorama photographing.

Further, an image photographing device and a control method thereof inaccordance with another embodiment may confirm differences betweenmoving velocity and direction of the image photographing device andreference velocity and direction during panorama photographing, thusallowing the image photographing device to be easily operated.

All references, including publications, patent applications, andpatents, cited herein are hereby incorporated by reference to the sameextent as if each reference were individually and specifically indicatedto be incorporated by reference and were set forth in its entiretyherein.

For the purposes of promoting an understanding of the principles of theinvention, reference has been made to the embodiments illustrated in thedrawings, and specific language has been used to describe theseembodiments. However, no limitation of the scope of the invention isintended by this specific language, and the invention should beconstrued to encompass all embodiments that would normally occur to oneof ordinary skill in the art. The terminology used herein is for thepurpose of describing the particular embodiments and is not intended tobe limiting of exemplary embodiments of the invention. In thedescription of the embodiments, certain detailed explanations of relatedart are omitted when it is deemed that they may unnecessarily obscurethe essence of the invention.

The apparatus described herein may comprise a processor, a memory forstoring program data to be executed by the processor, a permanentstorage such as a disk drive, a communications port for handlingcommunications with external devices, and user interface devices,including a display, touch panel, keys, buttons, etc. When softwaremodules are involved, these software modules may be stored as programinstructions or computer readable code executable by the processor on anon-transitory computer-readable media such as magnetic storage media(e.g., magnetic tapes, hard disks, floppy disks), optical recordingmedia (e.g., CD-ROMs, Digital Versatile Discs (DVDs), etc.), and solidstate memory (e.g., random-access memory (RAM), read-only memory (ROM),static random-access memory (SRAM), electrically erasable programmableread-only memory (EEPROM), flash memory, thumb drives, etc.). Thecomputer readable recording media may also be distributed over networkcoupled computer systems so that the computer readable code is storedand executed in a distributed fashion. This computer readable recordingmedia may be read by the computer, stored in the memory, and executed bythe processor.

Also, using the disclosure herein, programmers of ordinary skill in theart to which the invention pertains may easily implement functionalprograms, codes, and code segments for making and using the invention.

The invention may be described in terms of functional block componentsand various processing steps. Such functional blocks may be realized byany number of hardware and/or software components configured to performthe specified functions. For example, the invention may employ variousintegrated circuit components, e.g., memory elements, processingelements, logic elements, look-up tables, and the like, which may carryout a variety of functions under the control of one or moremicroprocessors or other control devices. Similarly, where the elementsof the invention are implemented using software programming or softwareelements, the invention may be implemented with any programming orscripting language such as C, C++, JAVA®, assembler, or the like, withthe various algorithms being implemented with any combination of datastructures, objects, processes, routines or other programming elements.Functional aspects may be implemented in algorithms that execute on oneor more processors. Furthermore, the invention may employ any number ofconventional techniques for electronics configuration, signal processingand/or control, data processing and the like. Finally, the steps of allmethods described herein may be performed in any suitable order unlessotherwise indicated herein or otherwise clearly contradicted by context.

For the sake of brevity, conventional electronics, control systems,software development and other functional aspects of the systems (andcomponents of the individual operating components of the systems) maynot be described in detail. Furthermore, the connecting lines, orconnectors shown in the various figures presented are intended torepresent exemplary functional relationships and/or physical or logicalcouplings between the various elements. It should be noted that manyalternative or additional functional relationships, physical connectionsor logical connections may be present in a practical device. The words“mechanism”, “element”, “unit”, “structure”, “means”, and “construction”are used broadly and are not limited to mechanical or physicalembodiments, but may include software routines in conjunction withprocessors, etc.

The use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate the inventionand does not pose a limitation on the scope of the invention unlessotherwise claimed. Numerous modifications and adaptations will bereadily apparent to those of ordinary skill in this art withoutdeparting from the spirit and scope of the invention as defined by thefollowing claims. Therefore, the scope of the invention is defined notby the detailed description of the invention but by the followingclaims, and all differences within the scope will be construed as beingincluded in the invention.

No item or component is essential to the practice of the inventionunless the element is specifically described as “essential” or“critical”. It will also be recognized that the terms “comprises,”“comprising,” “includes,” “including,” “has,” and “having,” as usedherein, are specifically intended to be read as open-ended terms of art.The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the invention (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless the context clearly indicates otherwise. In addition, itshould be understood that although the terms “first,” “second,” etc. maybe used herein to describe various elements, these elements should notbe limited by these terms, which are only used to distinguish oneelement from another. Furthermore, recitation of ranges of values hereinare merely intended to serve as a shorthand method of referringindividually to each separate value falling within the range, unlessotherwise indicated herein, and each separate value is incorporated intothe specification as if it were individually recited herein.

What is claimed is:
 1. An image photographing device, which performspanorama photographing by photographing a plurality of images and thencombining the plurality of images into a single panoramic image,comprising: a photographing unit configured to photograph the pluralityof images; an image processing unit configured to generate preview imagedata from the plurality of images photographed by the photographing unitand to generate preview panorama data using the preview image data; anda display unit configured to display both the preview image data and thepreview panorama data, wherein at least one of: a) a directionaldifference, indicating a relative difference between a moving directionof the image photographing device and a predetermined referencedirection, and b) a velocity difference, indicating a relativedifference between a moving velocity of the image photographing deviceand a predetermined reference velocity, is displayed on the displayunit; the device further comprising: a detection unit configured todetect a motion vector including data regarding moving direction orvelocity of the image photographing device during panoramaphotographing; and a control unit configured to stop panoramaphotographing when the moving direction of the image photographingdevice differs from the reference direction by at least a predeterminedangle.
 2. The image photographing device according to claim 1, furthercomprising a control unit configured to stop panorama photographing whenthe moving velocity of the image photographing device differs from thereference velocity by at least a predetermined extent.
 3. The imagephotographing device according to claim 1, wherein the control unitcontrols the image processing unit so that abnormal performance ofsynthesis of the preview panorama data is displayed when the imagephotographing device moves in a direction in which the synthesis of thepreview panorama data is not normally performed.
 4. The imagephotographing device according to claim 2, wherein the control unitcontrols the image processing unit so that abnormal performance ofsynthesis of the preview panorama data is displayed when the imagephotographing device moves at a velocity at which the synthesis of thepreview panorama data is not normally performed.
 5. A control method ofan image photographing device, which performs panorama photographing byphotographing a plurality of images and then combining the plurality ofimages into a single panoramic image, the control method comprising:photographing the plurality of images; converting the plurality ofimages into preview image data and generating preview panorama datausing the preview image data; and displaying both the preview image dataand the preview panorama data; displaying at least one of: a) adirectional difference, indicating a relative difference between amoving direction of the image photographing device and a predeterminedreference direction, and b) a velocity difference, indicating a relativedifference between a moving velocity of the image photographing deviceand a predetermined reference velocity; and calculating a motion vectorincluding data regarding moving direction or velocity of the imagephotographing device during panorama photographing; wherein panoramaphotographing is stopped when the moving direction of the imagephotographing device differs from the reference direction by at least apredetermined angle.
 6. The control method according to claim 5, whereinabnormal performance of synthesis of the preview panorama data isdisplayed when the image photographing device moves in a direction inwhich the synthesis of the preview panorama data is not normallyperformed.
 7. A control method of an image photographing device, whichperforms panorama photographing by photographing a plurality of imagesand then combining the plurality of images into a single panoramicimage, the control method comprising: photographing the plurality ofimages; converting the plurality of images into preview image data andgenerating preview panorama data using the preview image data; anddisplaying both the preview image data and the preview panorama data;displaying at least one of: a) a directional difference, indicating arelative difference between a moving direction of the imagephotographing device and a predetermined reference direction, and b) avelocity difference, indicating a relative difference between a movingvelocity of the image photographing device and a predetermined referencevelocity; calculating a motion vector including data regarding movingdirection or velocity of the image photographing device during panoramaphotographing; and stopping panorama photographing when the movingvelocity of the image photographing device differs from the referencevelocity by at least a predetermined extent.
 8. The control methodaccording to claim 7, wherein abnormal performance of synthesis of thepreview panorama data is displayed when the image photographing devicemoves at a velocity at which the synthesis of the preview panorama datais not normally performed.
 9. An electronic apparatus which is operableto photograph and arrange a plurality of images into a panoramic image,the apparatus comprising: a capturing unit configured to capture theplurality of images; an image processing unit configured to generatepreview live-view image data and to generate preview panorama imagedata; a detection unit configured to determine a current movingdirection of the electronic apparatus; and a display unit operable tosimultaneously display the preview live-view image data, the previewpanorama image data, and superimposedly display, over the previewpanorama image data, information relating to the current detected movingdirection of the electronic apparatus.
 10. The electronic apparatusaccording to claim 9, wherein a reference line is also simultaneouslydisplayed on the display unit.
 11. The electronic apparatus according toclaim 10, wherein the current detected moving direction of theelectronic apparatus is displayed relative to the reference line. 12.The electronic apparatus according to claim 9, wherein a relativedifference between a reference moving direction for panoramaphotographing and the current detected moving direction of theelectronic apparatus is also simultaneously displayed on the displayunit.
 13. The electronic apparatus according to claim 12, wherein therelative difference indicates a vertical deviation of the detectedmoving direction of the electronic apparatus from the reference movingdirection for panorama photographing.
 14. The electronic apparatusaccording to claim 9, wherein the detection unit determines a movingvelocity of the electronic apparatus and the display unit graphicallydisplays the detected moving velocity.
 15. The electronic apparatusaccording to claim 9, wherein the image processing unit generates thepreview panorama image data by downsizing the preview live-view imagedata and arranging the downsized preview live-view image data.
 16. Theelectronic apparatus according to claim 12, further comprising a controlunit configured to stop panorama photographing when the relativedifference exceeds a predetermined angle.
 17. An electronic apparatuswhich is operable to photograph and arrange a plurality of images into apanoramic image, the apparatus comprising: a capturing unit configuredto capture the plurality of images; an image processing unit configuredto generate preview live-view image data and to generate previewpanorama image data; a detection unit configured to determine a currentmoving direction of the electronic apparatus; and a display unitoperable to simultaneously display the preview live-view image data, thepreview panorama image data, and superimposedly display, over thepreview live-view image data, information relating to the currentdetected moving direction of the electronic apparatus.
 18. Theelectronic apparatus according to claim 9, which is further operable ina general photographing mode.
 19. A hand-held portable electronic devicewhich is operable to guide a user to move the device for capturing andarranging a plurality of images into a panoramic image, the devicecomprising: a photographing unit configured to capture the plurality ofimages, an image processing unit configured to generate previewlive-view image data and to generate preview panorama image data; adetection unit configured to detect current movement of the device; adisplay unit operable to display the preview live-view image data, thepreview panorama image data and superimposedly display, over the previewpanorama image data, information relating to the current detectedmovement of the device.
 20. The device according to claim 19, wherein areference line for panorama photographing is displayed on the displayunit.
 21. The device according to claim 20, wherein the current detectedmovement direction of the device is displayed relative to the referenceline.
 22. The device according to claim 19, wherein the current detectedmovement includes at least one of a current moving direction and acurrent moving velocity.
 23. A hand-held portable electronic devicewhich is operable to guide a user to move the device for capturing andarranging a plurality of images into a panoramic image, the devicecomprising: a photographing unit configured to capture the plurality ofimages, an image processing unit configured to generate previewlive-view image data and to generate preview panorama image data; adetection unit configured to detect current movement of the device; adisplay unit operable to display the preview live-view image data, thepreview panorama image data and superimposedly display, over the previewlive-view image data, information relating to the current detectedmovement of the device.
 24. A method of operating an electronic devicefor capturing and arranging a plurality of images into a panoramicimage, the method comprising: capturing the plurality of images;generating preview live-view image data; generating preview panoramaimage data; detecting a current moving direction of the electronicdevice; and displaying simultaneously the preview live-view image data,and preview panorama image data and superimposedly display, over thepreview panorama image data, information relating to the currentdetected moving direction of the electronic device.
 25. A method ofoperating an electronic device for capturing and arranging a pluralityof images into a panoramic image, the method comprising: capturing theplurality of images; generating preview live-view image data; generatingpreview panorama image data; detecting a current moving direction of theelectronic device; and displaying simultaneously the preview live-viewimage data, and preview panorama image data and superimposedly display,over the preview live-view image data, information relating to thecurrent detected moving direction of the electronic device.
 26. Themethod according to claim 24, further comprising displaying a referenceline for panorama photographing.
 27. The method according to claim 26,further comprising displaying the current detected moving direction ofthe electronic device relative to the reference line.
 28. The methodaccording to claim 24, wherein the preview panorama image data isgenerated by downsizing and combining the preview live-view image data.29. The method according to claim 24, further comprising detecting acurrent moving velocity of the electronic device and displayinginformation relating to the current detected moving velocity.